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(54) TiUe: DIRECTING IMAGE CAPTURE SEQUENCES IN A DIGITAL IMAGING DEVICE USING SCRIPTS 



(57) Abstract 

A method and system for control- 
ling user interaction in a digital imag- 
ing device that includes a display screen. 
The method and system include provid- 
ing the digital imaging device with a di- 
rected image capture sequence (618) that 
comprises a set of program instructions, 
which when executed, display interac- 
tive instructions on a display screen that 
prompt the user to perform specific opera- 
dons. In response to the user performing 
these operations, the interactive instruc- 
tions are automatically updated to thereby 
guide the user through a series of related 
image captures. The directed image cap- 
ture sequence is preferably written in an 
interpreted scripting language (610) and 
loaded into the camera from an external 
source to add functionality to the digital 
imaging device. 
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DIRECTING IMAGE CAPTURE SEQUENCES IN A 
DIGITAL IMAGING DEVICE USING SCRIPTS 

CROSS-REFERENCE TO RELATED APPLICATIONS 

This application is related to U.S. Patent Application No. , entitled "A 

Method And System For Controlling User Interaction In A Digital Imaging Device 

Using Dynamic Overlay Bars" (P135), and U.S. Patent Application No. , 

entitled "Method and System For Displaying Overlay Bars In A Digital Imaging 
Device" (P166), which were filed on the same date as the present application. 

FIELD OF THE INVENTION 

The present invention relates generally to digital imaging devices, 
including digital cameras, and more particularly to a method and system for 
controlling user interaction in a digital imaging device using dynamic overlay 
bars. 

BACKGROUND OF THE INVENTION 

Most digital cameras today are similar in size to and behave like 
conventional point-and-shoot cameras. Unlike conventional cameras, however, 
most digital cameras store digital images in an internal flash memory or on 
external memory cards, and some are equipped with a liquid-crystal display 
(LCD) screen on the back of the camera. Through the use of the LCD, most 
digital cameras operate in two modes, record and play, although some only 
have a record mode. 

In record mode, which is also referred to as capture mode, the LCD acts 
as a live viewfmder in which the user may view an object or scene before taking 
a picture, similar to the LCD on a camcorder. When the user presses the shutter 
button, whatever scene is shown on the LCD is captured as a still image. 
Besides capturing still images, some digital cameras can be set to capture other 
image types, such as burst and time-lapse images. A burst image is a series of 
still images captured in rapid succession, while a time-lapse image is series of 
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still images taken at regular intervals over a longer time period. 

In play mode, the LCD acts as a playback screen for reviewing the 
previously captured images. Typically, several small images are displayed on 
the LCD at once, and by selecting one of the images the user may then display 
the full-sized version of the images in the LCD. 

Although conventional digital cameras are more convenient for the user to 
use than film cameras due to instant play back of captured images, there are 
several drawbacks in the user interface that restrict user interaction with the 
camera. When capturing images, for example, it is often helpful for the user to 
be informed about the current settings or operational state of the camera, such 
as whether the flash is on/off, and the current image type setting, for instance. 

In conventional digital cameras, such status information is typically 
displayed as text blocks or accessed through a status screen or the like. The 
disadvantage with the text blocks is that they are typically small (10-15 
characters in length), and therefore, the amount of status information they can 
provide is very limited. Typically, text blocks are used to display information 
such as the current image number. Moreover, when text blocks are displayed 
with a solid color background, the background obscures that portion of the 
image. And when text blocks are displayed with no background (only text), the 
text is difficult to distinguish from the colors comprising the image, making the 
text hard to read. 

The disadvantage with status screens is that in order to view the status 
information, the image currently displayed on the LCD must be replaced with the 
status screen, causing the user to loose sight of the image. Another approach 
would be to shrink the display area of the LCD and add a black status area in 
the viewfinder, as done in optical viewfinders of film cameras. This, however, 
would shrink the size of images displayed in the viewfinder. 

Another drawback with conventional digital cameras is that as 
technological advances are made, digital cameras are continually provided with 
more features and functions, which make them more complex for the user to 
interact with. This is similar to what occurs with PC software, which increasingly 
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grows larger and harder to use. PC developers attempt to alleviate this problem 
by providing more and larger help menus. Each help menu usually opens in its 
own window with paragraphs of scrolling text. 

Using PC help menus in a digital camera to guide user interaction through 
the camera features and functions would be less than ideal because of the 
limited size of the camera LCD. And assuming help menus were displayed, they 
would either obscure whatever image was being displayed or otherwise total 
replace it, which is disadvantageous to the picture taker. 

Accordingly, what is needed is an improved system and method for 
displaying status information in a manner that does not obscure the display of 
the current object in the LCD, and for controlling user interaction in a digital 
imaging device. The present invention addresses such a need. 

SUMMARY OF THE INVENTION 

The present invention provides a method and system for controlling user 
interaction in a digital imaging device having a display using dynamic overlay 
bars. The digital imaging device includes at least two operating modes, where 
each of the operating modes has at least one mode-specific operation that can 
be performed on images. In response to operating in either of the operating 
modes, the digital imaging device displays a translucent overlay bar on the 
display that is dynamically updated with status information and interactive 
instructions that guide the user through the mode-specific operations. 

In a second aspect of the present invention, the interactive instructions 
are implemented using a script, which is a text-based program that may be 
easily written by the user and externally loaded into the camera. Once loaded 
into the camera, the commands comprising the script are translated and 
executed one-by-one by a script interpreter to guide the user through the newly 
provided function. 

A third aspect of the present invention, provides a method and system for 
displaying overlay bars on the display. First, text and graphic information to be 
displayed on the overlay bars are stored in an overlay bar buffer, and then 
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displayed on the display. Thereafter, the current image is displayed on the 
display line-by-line. The lines of the image that will be displayed within the area 
of an overlay bar are stored in a backstore buffer. Each line in the backstore 
buffer is merged with its corresponding lines in the overlay bar buffer and 
displayed. This aspect of the present invention makes the overlay bars appear 
translucent, and the image appear as though it is sliding beneath the overlay 
bars as it is being displayed. When the user turns-off the overlay bars, only the 
portions of the image stored in the backstore buffer need be re-displayed to 
provide the original image, thus eliminating the need to re-display the entire 
image. 

Accordingly, the method and system of the present invention provides 
status information to a user and allows the user to perform complex camera 
functions and features to the images with minimum effort, while allowing for easy 
viewing of the images. Displaying interactive instructions on dynamic overlay 
bars to guide the user through complex tasks in accordance with the present 
invention eliminates the need for help screens and for the user to remember 
complicated key sequences, and increases the ease of use and operation of the 
digital camera. The manner in which the overlay bars and the image is 
displayed makes the user interface more aesthetically pleasing, while increasing 
the display speed of the digital imaging device. 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a block diagram of a digital camera that operates in accordance 
with the present invention. 

FIG. 2 is a block diagram of an example embodiment for the imaging 
device of FIG. 1 . 

FIG. 3 is a block diagram of an example embodiment for the computer of 

FIG. 1. 

FIGS. 4 and 5 are diagrams depicting the preferred embodiment of the 
camera's 110 user interface. 

FIG. 6 is a flow chart is shown, illustrating the process of controlling user 
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interaction in a digital imaging device using dynamic overlay bars in accordance 
with the present invention 

FIGS. 7A and 7B are diagrams illustrating the use of dynamic overlay 
bars on the LCD screen during capture and play modes, respectively. 

FIGS. 8A through 8C are diagrams illustrating how the overlay bars may 
be used to guide the user through a recording of a sound annotation. 

FIGS. 9A and 9B are diagrams illustrating example directed image 
capture screens. 

FIG. 10 is a block diagram illustrating the camera software, which is 
stored in ROM, and DRAM, where the software is executed. 

FIG. 1 1 is a flow chart illustrating an exemplary process of installing and 
running a script-based directed image capture in a preferred embodiment of the 
present invention. 

FIG. 12A is a diagram illustrating a memory buffer organization for 
displaying overlay bars. 

FIG. 12B is a flow chart illustrating the process of displaying overlay bars 
on the LCD in accordance with the present invention. 

DETAILED DESCRIPTION OF THE INVENTION 

The present invention relates to an improved method and system for 
controlling user interaction in a digital imaging device using dynamic overlay 
bars. The following description is presented to enable one of ordinary skill in the 
art to make and use the invention and is provided in the context of a patent 
application and its requirements. Although the present invention will be 
described in the context of a digital camera, various modifications to the 
preferred embodiment will be readily apparent to those skilled in the art and the 
generic principles herein may be applied to other embodiments. That is, any 
digital imaging device which displays images, icons and/or other items, could 
incorporate the features described herein below and that device would be within 
the spirit and scope of the present invention. Thus, the present invention is not 
intended to be limited to the embodiment shown but is to be accorded the widest 
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scope consistent with the principles and features described herein. 

The present invention is a method and system for controlling user 
interaction in a digital imaging device using dynamic overlay bars. According to 
the present invention, both status information and interactive instructions are 
displayed on dynamic overlay bars to enable a user to perform complex camera 
functions and apply features to the images with minimum effort, while allowing 
for easy viewing of the images. 

Referring now to FIG. 1 , a block diagram of a digital camera 1 10 is shown 
for use in accordance with the present invention. Camera 110 preferably 
comprises an imaging device 114, a system bus 116 and a computer 118. 
Imaging device 114 is optically coupled to an object 1 12 and electrically coupled 
via system bus 116 to computer 118. Once a photographer has focused 
imaging device 114 on object 112 and, using a capture button or some other 
means, instructed camera 1 10 to capture an image of object 112, computer 118 
commands imaging device 114 via system bus 116 to capture raw image data 
representing object 112. The captured raw image data is transferred over 
system bus 116 to computer 118 which performs various image processing 
functions on the image data before storing it in its internal memory. System bus 
116 also passes various status and control signals between imaging device 114 
and computer 118. 

Referring now to FIG. 2, a block diagram of an example embodiment of 
imaging device 114 is shown. Imaging device 114 typically comprises a lens 
220 having an iris, a filter 222, an image sensor 224, a timing generator 226, an 
analog signal processor (ASP) 228, an analog-to-digital (A/D) converter 230, an 
interface 232, and one or more motors 234. 

In operation, imaging device 114 captures an image of object 112 via 
reflected light impacting image sensor 224 along optical path 236. Image 
sensor 224, which is typically a charged coupled device (CCD), responsively 
generates a set of raw image data in CCD format representing the captured 
image 112. The raw image data is then routed through ASP 228, A/D converter 
230 and interface 232. Interface 232 has outputs for controlling ASP 228, 
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motors 234 and timing generator 226. From interface 232, the raw image data 
passes over system bus 1 16 to computer 118. 

Referring now to FIG. 3, a block diagram of an example embodiment for 
computer 118 is shown. System bus 116 provides connection paths between 
imaging device 114, an optional power manager 342, central processing unit 
(CPU) 344, dynamic random-access memory (DRAM) 346, input/output 
interface (I/O) 348, non-volatile memory 350, and buffers/connector 352. 
Removable memory 354 connects to system bus 116 via buffers/connector 352. 
Alternately, camera 110 may be implemented without removable memory 354 or 
buffers/connector 352. 

Power manager 342 communicates via line 366 with power supply 356 
and coordinates power management operations for camera 110. CPU 344 
typically includes a conventional processor device for controlling the operation of 
camera 110. In the preferred embodiment, CPU 344 is capable of concurrently 
running multiple software routines to control the various processes of camera 
110 within a multithreaded environment. DRAM 346 is a contiguous block of 
dynamic memory which may be selectively allocated to various storage 
functions. LCD controller 390 accesses DRAM 346 and transfers processed 
image data to LCD screen 402 for display. 

I/O 348 is an interface device allowing communications to and from 
computer 118. For example, I/O 348 permits an external host computer (not 
shown) to connect to and communicate with computer 118. I/O 348 also 
interfaces with a plurality of buttons and/or dials 404, and an optional status LCD 
406, which in addition to the LCD screen 402, are the hardware elements of the 
camera's user interface 408. 

Non-volatile memory 350, which may typically comprise a conventional 
read-only memory or flash memory, stores a set of computer-readable program 
instructions to control the operation of camera 110. Removable memory 354 
serves as an additional image data storage area and is preferably a non-volatile 
device, readily removable and replaceable by a camera 110 user via 
buffers/connector 352. Thus, a user who possesses several removable 
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memories 354 may replace a full removable memory 354 with an empty 
removable memory 354 to effectively expand the picture-taking capacity of 
camera 110. In the preferred embodiment of the present invention, removable 
memory 354 is typically implemented using a flash disk. 
5 Power supply 356 supplies operating power to the various components of 

camera 110. In the preferred embodiment, power supply 356 provides operating 
power to a main power bus 362 and also to a secondary power bus 364. The 
main power bus 362 provides power to imaging device 114, I/O 348, non-volatile 
memory 350 and removable memory 354. The secondary power bus 364 

1 0 provides power to power manager 342, CPU 344 and DRAM 346. 

Power supply 356 is connected to main batteries 358 and also to backup 
batteries 360. In the preferred embodiment, a camera 110 user may also 
connect power supply 356 to an external power source. During normal 
operation of power supply 356, the main batteries 358 provide operating power 

15 to power supply 356 which then provides the operating power to camera 1 1 0 via 

both main power bus 362 and secondary power bus 364. During a power failure 
mode in which the main batteries 358 have failed (when their output voltage has 
fallen below a minimum operational voltage level) the backup batteries 360 
provide operating power to power supply 356 which then provides the operating 

20 power only to the secondary power bus 364 of camera 110. 

FIGS. 4 and 5 are diagrams depicting the preferred hardware 
components of the camera's 110 user interface 408. FIG. 4 is back view of the 
camera 110 showing the LCD screen 402, a four-way navigation control button 
409, an overlay button 413, a menu button 414, and a set of programmable soft 

25 keys 416. FIG. 5 is a top view of the camera 110 showing a shutter button 418, 

and a mode dial 420. The camera may optionally include status LCD 406, 
status LCD scroll and select buttons 422 and 424, a sound record button 426, 
and zoom-in, zoom-out buttons 428a and 428b. 

The digital camera of the present invention is controlled by graphical- 

30 user-interface (GUI) based operating system (OS), which is in contrast to 

conventional digital cameras that are controlled by proprietary hardware 
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architectures. 

In the preferred embodiment of the present invention, the OS provides 
the digital camera with several different operating modes for supporting various 
camera functions. Although the digital camera may include several different 
operating modes, the modes relevant to this description are capture mode, and 
play mode. 

In capture mode, the camera 100 supports the actions of preparing to 
capture an image, and capturing an image through the use of either the LCD 
screen 402 or the status LCD 406. In play mode, the camera 110 supports the 
actions of displaying full-sized views of captured images, and play-backing 
various media types associated with the images, such as sound. The user may 
switch between the various modes, using the mode dial 420. When the camera 
is placed into a particular mode, that mode's default screen appears in the LCD 
screen 402 in which a set of mode-specific items, such as images, icons, and 
text, are displayed. 

The present invention provides a method and system for controlling user 
interaction in a digital imaging device using dynamic overlay bars. According to 
the present invention, the dynamic overlay bars are used to provide the user 
with both status information and interactive instructions. The interactive 
instructions are automatically updated in response to normal camera operations 
to guide the user through predefined operations of the camera, thus making the 
device extremely easy to use. In addition, the manner in which the dynamic 
overlay bars are displayed reduces viewing interference with the currently 
displayed object. 

Referring now to FIG. 6, a flow chart is shown illustrating the process of 
controlling user interaction in a digital imaging device using dynamic overlay 
bars in accordance with the present invention. The process begins by 
displaying an image on the LCD screen 402 along with at least one overlay bar 
that provides a dynamic prompt area in a way that minimizes viewing 
interference with the displayed image in step 450. 

In a preferred embodiment, viewing interference is minimized by 
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positioning the overlay bar along an edge of the LCD screen 402 and by 
displaying the background of the bar translucently so that the user may see the 
image through the overlay bar. The overlay bar may also be displayed with a 
solid color background, but this is less desirable since the bar would overwrite 
5 that portion of the image. 

In response to the camera being placed into one of the operating modes, 
the overlay bar displays mode-specific information for the user in step 452. In a 
preferred embodiment, the mode-specific information displayed on the overlay 
bar includes a combination of static status information, dynamically updated soft 

10 key labels, and interactive instructions pertaining to the particular mode, as 

described further below. After the mode-specific information is displayed, the 
mode-specific information is then dynamically updated during the operation of 
the camera to guide the user through a mode-specific function in step 454. 

To more particularly describe the present invention, refer to FIGS. 7A and 

1 5 7B illustrating the use of dynamic overlay bars on the LCD screen 402 during 

two different operating modes of the digital camera 110. As shown, in a 
preferred embodiment of the present invention, two overlay bars 430 and 432 
are simultaneously displayed on the LCD screen 402, rather than one, to strike a 
balance between the amount of information provided to the user and the amount 

20 of screen area consumed by text and/or graphics. 

Overlay bar 430 may be used primarily to display status information and 
interactive instructions, while overlay bar 432 may be used primarily to display 
soft key labels 410 corresponding to soft keys 412. Both overlay bars 430 and 
432 may be turned-off in each of the camera operating modes by pressing the 

25 overlay "on/off" button 413 so that users can have an unobstructed view of 

images if they so choose (off), or extra help in operating the camera (on). 

Referring to FIG. 7A, the display of the overlay bars 430 and 432 on the 
LCD screen 402 during capture mode is shown. In capture mode, the camera 
110 supports the actions of preparing to capture an image, and capturing an 

30 image through the use of either the LCD screen 402 alone or with the aid of an 

optional optical viewfinder (not shown). 
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Overlay bar 430 is updated with capture status information during capture 
mode, which may include a graphic memory gauge, and text indicating the state 
of the camera (Ready), for example. The memory gauge provides the user with 
a constant overview of camera memory usage in terms of disk space, and may 
also show working memory usage. In a preferred embodiment, the memory bar 
displays disk space usage as segments filling-up, and displays working memory 
usage as the bar below those segments, which is constantly updated to reflect 
current memory status. When the working memory buffers are empty, the 
bottom part of the bar would be clear. When there is the equivalent of storage 
for only a few pictures left, the storage gauge may flash and the overlay bar 430 
may be updated with a message, such as "Storage Almost Full". If a user tries 
to take a picture without adequate storage, then the overlay bar 430 may be 
updated to reflect this status by displaying the message "Inadequate Storage," 
along with an optional sound from the camera. 

The overlay bar 430 may also be updated to reflect other types of capture 
status information and may be expanded into additional lines if needed. The 
additional capture status information could include the following: 1) Low Battery 
Indication - when main batteries run low, a battery icon may replace the storage 
gauge and a overlay bar 430 may be updated to flash "Battery Low"; 2) Shake 
Warning Indication - when light level is too low for recommended hand held 
operation and user has disabled the strobe system "Shake Warning" may be 
displayed in the overlay bar 430; and 3) No Focus Indication - when the focus 
system cannot adequately focus the camera lens, a "No Focus" may be 
displayed in the overlay bar 430. 

Referring now to FIG. 7B, the display of the overlay bars 430 and 432 on 
the LCD screen 402 during play mode is shown. In a preferred embodiment, the 
play screen layout displays one full-sized image at a time and the user may 
chronologically scroll through the full-sized images in the LCD screen 402 using 
the left/right buttons on four-way navigation control button 409. Users can also 
play back various media types, such as time-lapse, bursts and slide show 
images according to either default or user defined play back rates. 
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In the play mode, overlay bar 430 displays status information relating to 
the current image being displayed, such as the image name/number, and the 
date and time of capture. The status information may also include graphical 
icons indicating what category of images the image belongs to and the image 
type. 

Referring to both FIGS. 7A and 7B. besides displaying status information, 
the second use of the dynamic overly bars of the present invention is to display 
soft key labels 410 for soft keys 412. As described in U.S. Patent Application 
No. 08/939,993 filed on September 26, 1997, entitled "A Method And System 
For Manipulating Images Stored In A Digital Imaging Device," assigned to the 
present assignee and hereby incorporated by reference, soft keys 412a. 412b, 
and 412c of the user interface 400 are programmable, i.e., they may be 
assigned predefined functions. The function currently assigned to a respective 
soft key 412 is indicated by the soft key labels 410a, 410b, and 410c displayed 
in overlay bar 432. After a soft key label 410 has been displayed, the user may 
then press the corresponding soft key 412 to have the function indicated by its 
label applied to the current image. 

Referring to FIG. 7B for example, the function assigned to the soft key 
412b in during play mode is a "Zoom" function, which allows a user to zoom in 
and out of a displayed image. When the user zooms-in on an image by pressing 
the soft key 412b, the "Zoom" soft key label 410b is changed to "Zoom-out". 
While an image is zoomed, the user may pan around the image using the four- 
way control button 406. 

The functions assigned to the soft keys 412, and thus the soft key labels 
410, are changed in response to several different factors. The soft keys 412 
may change automatically either in response to user actions, or based on 
predetermined conditions existing in the camera, such as the current operating 
mode, the image type, and so on. The soft keys 412 may also be changed 
manually by the user by pressing the menu button 415. Providing 
programmable soft keys 412 increases the number of functions that may be 
performed by the camera, while both minimizing the number of buttons required 
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on the user interface, and reducing the need to access hierarchical menus. 

As stated above, in addition to displaying status information and soft key 
labels, the dynamic overlay bars of the present invention may also be used to 
display interactive instructions to the user to guide user through camera 
5 functions. Basic types of camera functions include reviewing captured images, 

deleting images, annotating images with sound, and capturing groups of related 
images. With conventional cameras, the user would have to memorize 
complicated key sequences in order to perform these functions. 

The present invention, in contrast, uses the dynamic overlay bars to 
10 display interactive instructions that guide the user through operations such as 

adding sound to an image, deleting images and/or sound, and capturing groups 
of related images. 

As described in U.S. Patent Application No. 08/939,993, for example, after the 
user has captured an image and the image is displayed for review, the overlay 
15 bar 432 automatically reminds the user that he or she has the option to delete 

the image. That is, one of the soft key labels 410 is changed to "Delete" and the 
user may then delete image by pressing the corresponding "Delete" soft key 
412. 

Referring now to FIGS. 8A through 8C, diagrams illustrating how the 
20 overlay bars may be used to guide the user through a recording of a sound 

annotation are shown. The user may initiate the sound annotation function by 
pressing the record button 426 (see FIG. 5) while an image is displayed. In 
response, a record indication, such as a microphone icon, is automatically 
displayed in overlay bar 430 along with a display of the duration of the recording, 
25 as shown in FIG. 8A. After the sound annotation is recorded, the soft key labels 

410 may be updated to display three options "Play", "Delete", and "Save"; where 
"Play" plays back the recorded sound, "Delete" deletes the recorded sound, and 
tt Save n saves the recorded sound. 

If the user is reviewing images in play or review modes, it is possible that 
30 the displayed image will have a sound annotation attached. Should the -user 

presses the "Delete" soft key 412, it is unclear what operation the user wishes to 
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perform: delete the image, delete only the sound, or delete both. Indeed, an 
inexperienced user may not even consider all three of these possibilities before 
pressing the "Delete" button. Therefore, to guide the user through this 
operation, the dynamic overlay bars 430 and 432 are updated to prompt the 
user whether the image or the sound annotation is to be deleted, as shown in 
FIG. 8B. The user may then indicate which is to be deleted by pressing the 
corresponding soft key 412. 

While reviewing images, it is also possible that the user may press the 
record button 426. If the current image already includes a sound annotation, 
then it is unclear whether the user wishes to record a new sound annotation 
over the old one, or whether the user is unaware of the existing sound 
annotation. Therefore, to make sure the user doesn't inadvertently overwrite the 
existing sound, the overlay bar 430 is automatically updated to inform the user 
that sound will not be recorded until the user deletes the existing sound, as 
shown in FIG. 8C. In addition, if the user doesn't recall the contents of the 
previous sound annotation, the user may listen to it before deleting it by pressing 
"Play", or the user may cancel the record operation altogether by pressing "Exit". 
Thus, according to the present invention the user is enabled to perform complex 
tasks in the camera without fumbling through a set of hierarchical menus. 

Another use of displaying interactive instructions in the dynamic overlay 
bars 430 and 432 in accordance with the present invention is to direct the user 
through image capture sequences. The purpose of directed image capture 
sequences is to customize the camera's image capture process for a specific 
application. More specifically, a directed image capture is a camera feature that 
provides the user with interactive instructions and feedback during capture mode 
to guide the user through a series of task-oriented image captures. 

Upon initiation of a directed image capture sequence, interactive 
instructions are displayed the dynamic overlay bars 430and 432 that prompt the 
user to perform specific operations (capture image or capture sound), and for 
prompting the user to enter specific input (name and date). Customized directed 
image captures can be tailored to specific professions, such as insurance claims 
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adjusters and real estate agents, who would benefit from the use of a digital 
camera to capture groups of related pictures. 

Referring now to FIGS. 9A and 9B, diagrams illustrating example directed 
image capture screens are shown. The example shown in FIG. 9A may pertain 
to an insurance-related directed image capture that prompts ar> insurance claims 
adjuster to take a series of pictures of a damaged vehicle, or it may pertain to a 
real estate application that guides a user through taking photos of a house for 
sale. 

In the insurance example, once the directed image capture has started, 
the user may be instructed to take various views of the damaged car. The user 
may also be shown the number of the current image in that sequence, and the 
total number of images to be captured. 

After the views of the car are taken, the directed image capture may then 
prompt the user to enter specific information, such as the name of the image, as 
shown in FIG. 9B. The user may then enter text by choosing letters using the 
four-way control button 409. For insurance purposes, the directed image 
capture may also request the user to input the owner's name, license plate 
number, claim number, and so on. The sequence of images and corresponding 
information may then be downloaded from the camera or to a host computer for 
automated database storage or web page generation. 

In one embodiment of the present invention, one or more directed image 
capture sequences may be provided in the camera as built-in functions, 
especially if the camera is tailored for specific industries. 

However, in a second aspect of the present invention, the camera is 
made more flexible by implementing the directed image capture sequences as a 
set of program instructions that are externally loaded into the camera. Once 
loaded in the camera 110, the instructions are then preferably executed by the 
GUI-based system software running on CPU 344. 

FIG. 10 is a block diagram illustrating the contents of ROM 350 where the 
software is stored, and DRAM 346 where the software is executed. The 
software 600 may include a control application 602, a toolbox 604, drivers 612, a 
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kernel 614, and a startup/ configuration module 616. The control application 
602 is the main program that controls high-level functions of the digital camera 
and is responsible for interfacing with functions in the toolbox 604. 

Toolbox 604 comprises selected function modules that control how the 
digital camera captures and manipulates images. The modules may include 
image processors 606, a camera control shell 608, and a script interpreter 610. 
Image processors 606 are programs for enhancing (e.g., adjusting the contrast, 
sharpening, converting the image to gray-scale, etc.) the digital image received 
from imaging device 114. Camera control shell 608 receives and processes 
data structures for controlling camera functions. Script interpreter 610 translates 
and executes script statements, which are used to provide the directed image 
capture sequences and other camera 110 features, as explained below. 

Drivers 612 comprise program instructions for controlling various camera 
110 hardware components, such as motor 234 (FIG. 2) and a flash (not shown). 
Kernel 614 comprises program instructions providing basic underlying camera 
operating system services including synchronization routines, task creation, 
activation and deactivation routines, resource management routines, etc. 
Startup/configuration 616 comprises program instructions for providing initial 
camera 110 start-up routines such as the system boot routine and system 
diagnostics 

When the camera 110 is first turned on and booted up, the 
startup/configuration 616 module begins to execute and loads the drivers 612, 
the kernel 614, the control application 602, and system files containing 
configuration information into DRAM 346. Thereafter, operation of the camera is 
passed to the control application 602. In an alternative embodiment, the 
software 600 may executed out of ROM 350 in order to reduce the size of 
DRAM 346. 

The directed image capture sequence 618 may be loaded into the digital 
camera 110 from the removable memory 354 (FIG. 3), a host computer, or a 
network, and stored in DRAM 346 to run in place of the control application 602. 
In a preferred embodiment, the directed image capture sequence 618 is 
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implemented using a script, which is a program written with text-based 
commands that may be easily written by the user. As used herein, a script may 
be written in any intrepreted language, such as Basic and Lisp, for example. 

Once loaded into the camera, the script may be selected by the user from 
a menu where it is displayed for selection, and is thereafter executed by the 
control application 602 by passing the script to the script interpreter 610. The 
script interpreter 610 then translates and executes the script instructions 
comprising the directed image capture sequence 618 one-by-one. 

in an alternative embodiment, a directed image capture sequence 618 
may be implemented as a traditional application program, rather than a script. 
However, an application program is typically written by a software developer in a 
traditional computer language, such as C++, compiled, and stored in machine 
language, which is a more complicated process than adding new functions to 
the camera via a text-based interpreted script. 

FIG. 1 1 is a flow chart illustrating an exemplary process of installing and 
running a script-based directed image capture in a preferred embodiment of the 
present invention. The process begins by inserting the removable memory 354 
in step 700. When the removable memory 354 is installed, the removable 
memory 354 is mounted by the operating system 600 in step 702. Thereafter, 
the operating system searches for system files on the removable memory 354, 
which alert the digital camera 110 to the presence of an external program, in 
step 704. 

Any system files found on the removable memory 354 and corresponding 
directed image capture sequences 618 are then installed and made available to 
the user for selection via menu choices that appear on the LCD screen 402 in 
step 706. In a preferred embodiment, steps 704 and 706 are implemented as a 
hot-mount process when the removable memory 354 is inserted into the camera 

1 10, as described in U.S. patent application No. , entitled "Method 

And System For Dynamically Updating Software Functions In A Digital Capture 

Device (P149)," filed on , 1998, which is assigned to assignee of the 

present application and herein incorporated by reference. 



WO 99/44166 



PCT/US99/01973 



18 

Once the list of available directed image capture sequences 618 are 
displayed, the user selects one of the directed image capture sequences 618 to 
run in step 708. In a preferred embodiment, the list showing the available 
directed image capture sequences may be categorized in menus for easier 
selection. For example, assume a real estate agent has three different scripts 
for capturing images of different types of properties. The agent may name or 
create categories for the directed image capture sequences called "commercial", 
"industrial", and "residential", for instance. Selecting the residential category, for 
example, will cause a list of directed image captures to be displayed that are 
designed to capture pictures of different types of residential properties, such as 
one, two, and three bedroom homes. The user may then select a desired script 
depending on the particular house to be shot. 

In one preferred embodiment, the directed image capture selections 
displayed in the menus may be erased from the camera by rebooting the 
camera, or by removing the removable memory 354 from the camera 110. 

After the user selects one of the directed image capture sequences 618 
to run, the script interpreter 610 begins interpreting the directed image capture 
sequence 618 in step 710, and control is passed from the control application 
602 to the script. In step 712, the script interpreter 610 fetches the first 
command comprising the directed image capture sequence 618. 

It is then determined whether the fetched command is a script 
"WaitForShutter" command in step 714. This command causes control of the 
camera 110 to pass back to the control application 602 until the user presses 
the shutter button 418 to capture an image. The "WaitForShutter" command is 
preferably called with a quoted string parameter that is used in the dynamic 
overly bar 430 as the prompt to the user requesting an image capture (e.g., 
"Take photo of kitchen"). 

If the command is a "WaitForShutter* command in step 714, then control 
is returned to the script after the user presses the shutter button 418 in step 716 
to capture an image. If the fetched command is not a "WaitForShutter" 
command in step 714, then the script interpreter 610 interprets and executes the 
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command in step 718. 

After the user presses the shutter button 418 or after a script command 
has been executed, It is determined if the end of the script has been reached in 
step 720. If not, then the next command is fetched in step 712, and the process 
continues until the end of the script is reached, at which point control is returned 
to the control application 602 in step 722. 

Besides the "WaitForShutter" command, scripts may include, two other 
categories of script commands. One category of commands pertain to camera 
settings, controls and other camera parameters specific to the subject and/or 
scene being captured, (ie: White Balance Modes, Exposure Modes, and Focus 
Modes). This category of commands enable users to input "Hints" optimizing the 
camera's photo systems for specific photographic conditions. 

The other category of commands may pertain to file system operations 
and image tagging functions specific to the way in which image data is stored in 
memory, (ie: Guided Capture, Prompted Text/Audio Annotation, and Automated 
Image Grouping/Cataloging/lndexing.) This category of commands is 
particularly useful when used in conjunction with desktop computer applications 
where the hosting application is coordinated to take advantage of the 
Preformatted media organization and tag information. For example, while a 
directed image capture sequence guides the user though a series of steps to 
create an image grouping, the script commands comprising the sequence 
generate appropriate tags and data structures to group the images and text 
captured during the sequence. 

No matter whether the dynamic overlay bars of the present invention are 
used to display status information, soft key labels, or interactive instructions, as 
described herein, one important component affecting the user's experience is 
the method used to display the overlay bars on the image. 

One approach would be the follow prior art techniques for displaying text 
(e.g. image name) over an image. This approach typically includes the following 
steps: 1) fetching the image to be displayed, which is typically stored in -JPEG 
format, 2) decompressing and resizing the image, 3) displaying the 
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decompressed image block-by-block, and then after the image is fully displayed, 
4) writing the text on top of the image. 

The problem with this method is that is visually unappealing to the user, 
and it reduces the performance of camera when the user turns-off the text 
display while viewing the image. The reason the method reduces camera 
performance is the following. When text or graphics are displayed over the 
image, they obscure a portion of the image. And when the text is turned-off, the 
obscured portions of the image must be displayed so that the original image is 
seen without the text. In order to do this, however, the entire JPEG image must 
be fetched and decompressed again so that the obscured portions of the image 
can be displayed on the LCD, which can be a time consuming operation. 

A third aspect of the present invention overcomes these disadvantages 
by providing an improved method and system for displaying the overlay bars that 
not only enhances the visual effect associated with the overlay bars, but also 
eliminates the need to re-decompress the JPEG image data when the user 
turns-off the overlay bars, thereby increasing performance of the camera. 

According to this aspect of the present invention, the overlay bars are 
displayed first, followed by the image, wherein the image is made to appear as 
though it is sliding underneath the overlay bars as it is being displayed. The 
image appears as though is it is sliding underneath the overlay bars because the 
image is displayed on the LCD screen 402 line-by-line or block-by-block (as 
used herein, a block may include anywhere from one line to sixteen lines of 
image data). As the display of the image progresses from the top of the screen 
402, the image therefore appears to be displayed behind the overlay bars 430 
and 432 which are already present on the LCD screen 402. 

The overlay bars 430 and 432 are also provided with a translucent 
background so that so that the overlay bars 430 and 432 themselves do not 
obscure the image, but the text is easily distinguishable from the colors of the 
displayed image. The result is that after the image has been displayed, the 
overlay bars appear as a separate layer over the image. Further, the portions of 
the original image that intersect with the overlay bars 430 and 432 are saved, so 
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that when the user turns-off the overlay bars 430 and 432, only these portions of 
the image are redisplayed to restore the image. Thus this aspect of the present 
invention eliminates the need to re-decompress and display the entire image 
again, thereby increasing system performance. 

Where typically, specialized hardware would be required to achieve the 
above-described effects, the present invention accomplishes the task through 
software and the manipulation of several memory buffers, as shown in FIG. 12A. 

FIG. 12A is a diagram illustrating a buffer organization for displaying 
overlay bars, which in a preferred embodiment resides in DRAM 346. The 
buffer organization includes an overlay bar buffer 540, a backstore buffer 542, 
and a display buffer 544. According to the present invention, the overlay bar 
buffer 540 is used to store the graphics data (graphics and text) that will be 
displayed in the overlay bars 430 and 432. In a preferred embodiment the 
overlay bar buffer 540 is divided into a top and bottom portion, which store 
twenty lines of data each that correspond to the top and bottom overlay bar 430 
and 432, respectively. 

The backstore buffer 542 is used to store original image data 
corresponding to the area of the LCD screen 402 where the overlay bars 430 
and 432 will be displayed. The backstore buffer 542 is also divided into a top 
and bottom portion that are the same size as the top and bottom portions of the 
overlay bar buffer 540. 

As is typical in most rendering systems, the display buffer 544 is used to 
store the actual data that is to be displayed on the LCD. The data in the display 
buffer is accessed by LCD controller 390 (FIG. 3) and displayed on the LCD. 

FIG. 12B is a flow chart illustrating the process of displaying overlay bars 
on the LCD in accordance with the present invention. The first step in the 
process is to preferably receive an input line of decompressed image data from 
an image processing system in step 800. The process may also be modified to 
receive an input block of decompressed image data. In a preferred 
embodiment, the image processing system for providing the input data -may 
include an image decompressor for decompressing the image data, and a 
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resizer for resizing the lines of image data to fit the size of LCD screen 402. 

Next, it is determined whether the overlay bars 430 and 432 are turned- 
on or off in step 802. If the overlay bars are turned-off, then the line of image 
data is copied directly to the display buffer 544 in step 804 for display on the 
5 LCD screen 402 and the process continues. If the overlay bars remain off for 

the duration of the time it takes to display the image line-by-line or block-by- 
block, then the entire image is displayed on the LCD screen 402 using only the 
display buffer 544. 

If the overlay bars are turned-on in step 802, then it is determined 

1 0 whether the line of data will be displayed within the area of the LCD screen 402 

that is occupied by an overlay bar in step 806. If the line is within an overlay 
bar, the line is copied into the backstore buffer 542 in step 808. The purpose of 
copying the line to the backstore buffer 542 is to save the portion of the image 
that will be displayed underneath the overlay bars 430 and 432. 

15 After the current line of image data is copied into the backstore buffer 

542, the corresponding line stored in the overlay bar buffer 540 is merged with 
the current line in the backstore buffer 542 in step 810. The purpose of merging 
the two lines is to display the background of the overlay bars 430 and 432 
translucently over the image on the LCD screen 402. This is done by halving 

20 the luminance value of each pixel of the image data from the backstore buffer 

542 that falls within the bounds of an overlay bar 430 or 432, and overwriting 
each pixel in the line of image data that falls under a pixel of text or graphic data 
from the overlay bar buffer 540. Halving the luminance value of the image data 
causes the colors of the image that overlap an overlay bar 430 or 432 to be half 

25 as bright, thus giving the overlay bar 430 or 432 a translucent appearance and 

allowing the user to see the image through the overlay bar 430 or 432, as shown 
in FIG. 7B. In an alternative embodiment, the translucency of the overlay bars 
430 and 432 is provided by increasing, rather than decreasing, the luminance 
value of each image pixel falling within the area of an overlay bar. In this case, 

30 the text displayed in the overlay bars 430 and 432 is displayed using a" dark 

color. 
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As the line from the overlay bar buffer 540 is merged with the line from 
the backstore buffer 542, the resulting merged line is written into the display 
buffer 544 for display in step 812. If the current line is the last line of image data 
in step 814, then the process ends. Otherwise the next line of image data is 
received in step 800 and the process continues. In an alternate embodiment of 
the present invention, the determination of whether the overlay bars 802 are 
on/off in step 802 may be performed after copying the input line to the backstore 
buffer 542 in step 8. In this embodiment, the input line is copied into the 
backstore buffer 542 even when the overlay bars 430 and 432 are off. 

In a preferred embodiment of present invention, the software 600 
controlling the digital camera 110 is implemented as event driven software, 
which responds to input from the user (select menu, press button, etc.) or other 
applications at unregulated times. When, for example, the user first switches to 
play mode and/or selects a new image to display, the first steps that are 
performed in the process are to blank the LCD screen 402, fill the overlay bar 
buffer 540 with relevant mode-specific information, and then contents of the 
overlay bar buffer 540 and the backstore buffer 542 are merged and written to 
the display buffer 544. In this case, the backstore buffer 542 may contain black 
or white pixel values to provide the blank screen. Thereafter, the process 
proceed as described in FIG. 13. 

If the user turns-off the overlay bars 430 and 432 while an image is 
displayed, then the process is interrupted and software 600 copies the entire 
contents of the backstore buffer 542, which contains the original image data, to 
the display buffer 544 for display. This causes the overlay bars to disappear 
from the LCD screen 402 and restores the original image without having to re- 
decompress and display the entire image over again. 

If the user then turns-on the overlay bars 430 and 432, the software 600 
merges the contents of the overlay bar buffer 540 and the backstore buffer 542 
to provide the translucent bars and text over the image, and then copies the 
result to the display buffer 544 for display. This may be done by executing step 
812 and 814 for each line of the data in the buffers 540 and 542. 
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Also, when the overlay bars 430 and 432 are on, if the overlay bars 430 
and 432 are updated by the control application 602 due to a change in status or 
instructions, the contents of the overlay bar buffer 540 and the backstore buffer 
542 are remerged and written into the display buffer 544 for display. 

A method and system for controlling user interaction in a digital imaging 
device using dynamic overlay bars has been disclosed, which enables a user to 
apply camera functions and features to images with minimum effort, while 
allowing for easy viewing of the image. In addition, the present invention 
displays dynamic interactive instructions to the user in the form of directed 
image capture to guide the user through complex task, without the need for help 
screens or for the user to remember complicated key sequences. Finally, the 
present invention provides a method for displaying overlay bars that eliminates 
the need to re-decompress and display the image when the user turns-off the 
overlay bars, which increases the responsiveness of the camera. 

Although the present invention has been described in accordance with 
the embodiments shown, one of ordinary skill in the art will readily recognize that 
there could be variations to the embodiments and those variations would be 
within the spirit and scope of the present invention. For example, the functions 
assigned to the soft keys, the number of soft keys, and the placement of the soft 
keys and labels in and around the display may vary. The method and system may 
also be implemented in digital imaging devices having only two modes, but that 
have multiple navigation screens within the "play mode" Accordingly, many 
modifications may be made by one of ordinary skill in the art without departing 
from the spirit and scope of the appended claims. 
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CLAIMS 

What is claimed is: 

1 A method for controlling user interaction in a digital imaging device, 
the digital imaging device having a display the method comprising the steps of: 

a) providing the digital imaging device with a directed image capture 
sequence comprising a set of program instructions; 

b) executing the directed image capture sequence to display 
interactive instructions on the display that prompt the user to 
perform specific operations; and 

c) in response to the user performing the specific operations, 
automatically updating the interactive instructions, thereby guiding 
the user through a series of related image captures. 

2 The method of claim 1 wherein step a) further includes the step of 
providing the directed image capture sequence by externally loading the 
program instructions into the digital imaging device. 

3 The method of claim 2 wherein step a) further includes the step of 
providing the program instructions as a text-based script 

4 The method of claim 3 wherein step b) further includes the step of 
executing the directed image capture sequence by interpreting the text-based 
script 

5 The method of claim 4 wherein step c) further includes the step of 
prompting the user for specific information, and entering the specific information 
on a text entry screen. 

6 The method of claim 5 further includes the step of displaying the 
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interactive instructions on a translucent overlay bar. 

7 A method for directing image capture sequences in a digital 
imaging device having a display, the method comprising the steps of: 

a) externally loading a script comprising program commands into the 
digital imaging device; 

b) displaying the script as a menu item for selection by a user, 

c) in response to the user selecting the script menu item from the 
menu, passing operational control from the digital imaging device 
to a script interpreter; 

d) interpreting and executing each of the script commands, wherein a 
first plurality of the script commands are for displaying interactive 
instructions on the display requesting the user to perform specific 
camera operations, thereby guiding the user through a series of 
image captures; 

e) passing operational control from the script interpreter to the digital 
imaging device after the script has requested the user to capture 
an image; and 

f) repeating steps d-f until the script completes execution. 

8 The method of claim 7 wherein step a) further includes the step of loading 
the into the digital imaging device from a removable memory. 

9 The method of claim 8 wherein step e) further includes the step of 
passing operational control from the digital imaging device to the script 
after the user has captured the image. 

1 0 The method of claim 9 wherein step f) further includes the step of passing 
operational control back to the digital imaging device after the script 
completes execution. 
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11 An digital imaging device comprising: 
an imaging device for capturing image data; 

a memory coupled to the imaging device for storing the image data as captured 
images; 

a display for displaying a captured image; 

means for externally loading a script comprising program commands into the 
memory; 

a processor coupled to the imaging device and to the memory for controlling 
operation of the digital imaging device, the processor including means for 
interpreting and executing each of the script commands, wherein when the script 
commands are executed, interactive instructions are displayed on the display 
requesting the user to perform specific camera operations, thereby guiding the 
user through a series of related image captures. 

12 A digital imaging device as in claim 1 1 wherein the means for 
interpreting and executing each of the script commands comprises a script 
interpreter. 

13 A digital imaging device as in claim 12 wherein operation control of 
the digital imaging device is controlled by a control program stored in memory 
and executed by the processor, the control program comprising, 

means for displaying the script as a menu item on the display for selection by a 
user; and 

means for passing operational control to the script interpreter in response to the 
user selecting the script menu item from the menu. 
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DIRECTING IMAGE CAPTURE SEQUENCES IN A 
DIGITAL IMAGING DEVICE USING SCRIPTS 



CROSS-REFERENCE TO RELATED APPLICATIONS 

This application is related to U.S. Patent Application No. , entitled "A 

Method And System For Controlling User Interaction In A Digital Imaging Device 

Using Dynamic Overlay Bars" (P135). and U.S. Patent Application No. 

entitled "Method and System For Displaying Overlay Bars In A Digital Imaging 
Device" (P166), which were filed on the same date as the present application. 

FIELD OF THE INVENTION 

The present invention relates generally to digital imaging devices, 
including digital cameras, and more particularly to a method and system for 
controlling user interaction in a digital imaging device using dynamic overlay 
bars. 

BACKGROUND OF THE INVENTION 

Most digital cameras today are similar in size to and behave like 
conventional point-and-shoot cameras. Unlike conventional cameras, however, 
most digital cameras store digital images in an internal flash memory or on 
external memory cards, and some are equipped with a liquid-crystal display 
(LCD) screen on the back of the camera. Through the use of the LCD, most 
digital cameras operate in two modes, record and play, although some only 

have a record mode. 

In record mode, which is also referred to as capture mode, the LCD acts 
as a live viewfinder in which the user may view an object or scene before taking 
a picture, similar to the LCD on a camcorder. When the user presses the shutter 
button, whatever scene is shown on the LCD is captured as a still image. 
Besides capturing still images, some digital cameras can be set to capture other 
image types, such as burst and time-lapse images. A burst image is a series of 
still images' captured in rapid succession, while a time-lapse image is series of 
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still images taken at regular intervals over a longer time period. 

In play mode, the LCD acts as a playback screen for reviewing the 
previously captured images. Typically, several small images are displayed on 
the LCD at once, and by selecting one of the images the user may then display 
the full-sized version of the images in the LCD. 

Although conventional digital cameras are more convenient for the user to 
use than film cameras due to instant play back of captured images, there are 
several drawbacks in the user interface that restrict user interaction with the 
camera. When capturing images, for example, it is often helpful for the user to 
be informed about the current settings or operational state of the camera, such 
as whether the flash is on/off, and the current image type setting, for instance. 

In conventional digital cameras, such status information is typically 
displayed as text blocks or accessed through a status screen or the like. The 
disadvantage with the text blocks is that they are typically small (10-15 
characters in length), and therefore, the amount of status information they can 
provide is very limited. Typically, text blocks are used to display information 
such as the current image number. Moreover, when text blocks are displayed 
with a solid color background, the background obscures that portion of the 
image. And when text blocks are displayed with no background (only text), the 
text is difficult to distinguish from the colors comprising the image, making the 
text hard to read. 

The disadvantage with status screens is that in order to view the status 
information, the image currently displayed on the LCD must be replaced with the 
status screen, causing the user to loose sight of the image. Another approach 
would be to shrink the display area of the LCD and add a black status area in 
the viewfinder, as done in optical viewfinders of film cameras. This, however, 
would shrink the size of images displayed in the viewfinder. 

Another drawback with conventional digital cameras is that as 
technological advances are made, digital cameras are continually provided with 
more features and functions, which make them more complex for the user to 
interact with. This is similar to what occurs with PC software, which increasingly 
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grows larger and harder to use. PC developers attempt to alleviate this problem 
by providing more and larger help menus. Each help menu usually opens in its 
own window with paragraphs of scrolling text. 

Using PC help menus in a digital camera to guide user interaction through 
the camera features and functions would be less than ideal because of the 
limited size of the camera LCD. And assuming help menus were displayed, they 
would either obscure whatever image was being displayed or otherwise total 
replace it, which is disadvantageous to the picture taker. 

Accordingly, what is needed is an improved system and method for 
displaying status information in a manner that does not obscure the display of 
the current object in the LCD, and for controlling user interaction in a digital 
imaging device. The present invention addresses such a need. 

SUMMARY OF THE INVENTION 

The present invention provides a method and system for controlling user 
interaction in a digital imaging device having a display using dynamic overlay 
bars. The digital imaging device includes at least two operating modes, where 
each of the operating modes has at least one mode-specific operation that can 
be performed on images. In response to operating in either of the operating 
modes, the digital imaging device displays a translucent overlay bar on the 
display that is dynamically updated with status information and interactive 
instructions that guide the user through the mode-specific operations. 

In a second aspect of the present invention, the interactive instructions 
are implemented using a script, which is a text-based program that may be 
easily written by the user and externally loaded into the camera. Once loaded 
into the camera, the commands comprising the script are translated and 
executed one-by-one by a script interpreter to guide the user through the newly 
provided function. 

A third aspect of the present invention, provides a method and system for 
displaying overlay bars on the display. First, text and graphic information to be 
displayed on the overlay bars are stored in an overlay bar buffer, and then 
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displayed on the display. Thereafter, the current image is displayed on the 
display line-by-line. The lines of the image that will be displayed within the area 
of an overlay bar are stored in a backstore buffer. Each line in the backstore 
buffer is merged with its corresponding lines in the overlay bar buffer and 
displayed. This aspect of the present invention makes the overlay bars appear 
translucent, and the image appear as though it is sliding beneath the overlay 
bars as it is being displayed. When the user turns-off the overlay bars, only the 
portions of the image stored in the backstore buffer need be re-displayed to 
provide the original image, thus eliminating the need to re-display the entire 
image. 

Accordingly, the method and system of the present invention provides 
status information to a user and allows the user to perform complex camera 
functions and features to the images with minimum effort, while allowing for easy 
viewing of the images. Displaying interactive instructions on dynamic overlay 
bars to guide the user through complex tasks in accordance with the present 
invention eliminates the need for help screens and for the user to remember 
complicated key sequences, and increases the ease of use and operation of the 
digital camera. The manner in which the overlay bars and the image is 
displayed makes the user interface more aesthetically pleasing, while increasing 
the display speed of the digital imaging device. 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a block diagram of a digital camera that operates in accordance 
with the present invention. 

FIG. 2 is a block diagram of an example embodiment for the imaging 
device of FIG. 1. 

FIG. 3 is a block diagram of an example embodiment for the computer of 

FIG. 1. 

FIGS. 4 and 5 are diagrams depicting the preferred embodiment of the 
camera's 110 user interface. 

FIG. 6 is a flow chart is shown illustrating the process of controlling user 
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interaction in a digital imaging device using dynamic overlay bars in accordance 
with the present invention 

FIGS. 7A and 7B are diagrams illustrating the use of dynamic overlay 
bars on the LCD screen during capture and play modes, respectively. 

FIGS. 8A through 8C are diagrams illustrating how the overlay bars may 
be used to guide the user through a recording of a sound annotation. 

FIGS. 9A and 9B are diagrams illustrating example directed image 
capture screens. 

FIG. 10 is a block diagram illustrating the camera software, which is 
stored in ROM, and DRAM, where the software is executed. 

FIG. 1 1 is a flow chart illustrating an exemplary process of installing and 
running a script-based directed image capture in a preferred embodiment of the 
present invention. 

FIG. 12A is a diagram illustrating a memory buffer organization for 
displaying overlay bars. 

FIG. 12B is a flow chart illustrating the process of displaying overlay bars 
on the LCD in accordance with the present invention. 

DETAILED DESCRIPTION OF THE INVENTION 

The present invention relates to an improved method and system for 
controlling user interaction in a digital imaging device using dynamic overlay 
bars. The following description is presented to enable one of ordinary skill in the 
art to make and use the invention and is provided in the context of a patent 
application and its requirements. Although the present invention will be 
described in the context of a digital camera, various modifications to the 
preferred embodiment will be readily apparent to those skilled in the art and the 
generic principles herein may be applied to other embodiments. That is, any 
digital imaging device which displays images, icons and/or other items, could 
incorporate the features described herein below and that device would be within 
the spirit and scope of the present invention. Thus, the present invention is not 
intended to be limited to the embodiment shown but is to be accorded the widest 
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scope consistent with the principles and features described herein. 

The present invention is a method and system for controlling user 
interaction in a digital imaging device using dynamic overlay bars. According to 
the present invention, both status information and interactive instructions are 
displayed on dynamic overlay bars to enable a user to perform complex camera 
functions and apply features to the images with minimum effort, while allowing 
for easy viewing of the images. 

Referring now to FIG. 1 , a block diagram of a digital camera 110 is shown 
for use in accordance with the present invention. Camera 110 preferably 
comprises an imaging device 114, a system bus 116 and a computer 118. 
Imaging device 114 is optically coupled to an object 1 12 and electrically coupled 
via system bus 116 to computer 118. Once a photographer has focused 
imaging device 114 on object 112 and, using a capture button or some other 
means, instructed camera 110 to capture an image of object 112, computer 118 
commands imaging device 114 via system bus 116 to capture raw image data 
representing object 112. The captured raw image data is transferred over 
system bus 116 to computer 118 which performs various image processing 
functions on the image data before storing it in its internal memory. System bus 
116 also passes various status and control signals between imaging device 114 
and computer 118. 

Referring now to FIG. 2, a block diagram of an example embodiment of 
imaging device 114 is shown. Imaging device 114 typically comprises a lens 
220 having an iris, a filter 222, an image sensor 224, a timing generator 226, an 
analog signal processor (ASP) 228, an analog-to-digital (A/D) converter 230, an 
interface 232, and one or more motors 234. 

In operation, imaging device 114 captures an image of object 112 via 
reflected light impacting image sensor 224 along optical path 236. Image 
sensor 224, which is typically a charged coupled device (CCD), responsively 
generates a set of raw image data in CCD format representing the captured 
image 112. The raw image data is then routed through ASP 228, A/D converter 
230 and interface 232. Interface 232 has outputs for controlling ASP 228, 
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motors 234 and timing generator 226. From interface 232, the raw image data 
passes over system bus 1 1 6 to computer 118. 

Referring now to FIG. 3 ( a block diagram of an example embodiment for 
computer 118 is shown. System bus 116 provides connection paths between 
imaging device 114, an optional power manager 342, central processing unit 
(CPU) 344, dynamic random-access memory (DRAM) 346, input/output 
interface (I/O) 348, non-volatile memory 350, and buffers/connector 352. 
Removable memory 354 connects to system bus 116 via buffers/connector 352. 
Alternately, camera 110 may be implemented without removable memory 354 or 
buffers/connector 352. 

Power manager 342 communicates via line 366 with power supply 356 
and coordinates power management operations for camera 110. CPU 344 
typically includes a conventional processor device for controlling the operation of 
camera 110. In the preferred embodiment, CPU 344 is capable of concurrently 
running multiple software routines to control the various processes of camera 
110 within a multithreaded environment. DRAM 346 is a contiguous block of 
dynamic memory which may be selectively allocated to various storage 
functions. LCD controller 390 accesses DRAM 346 and transfers processed 
image data to LCD screen 402 for display. 

I/O 348 is an interface device allowing communications to and from 
computer 118. For example, I/O 348 permits an external host computer (not 
shown) to connect to and communicate with computer 118. I/O 348 also 
interfaces with a plurality of buttons and/or dials 404, and an optional status LCD 
406, which in addition to the LCD screen 402, are the hardware elements of the 
camera's user interface 408. 

Non-volatile memory 350, which may typically comprise a conventional 
read-only memory or flash memory, stores a set of computer-readable program 
instructions to control the operation of camera 110. Removable memory 354 
serves as an additional image data storage area and is preferably a non-volatile 
device, readily removable and replaceable by a camera 110 user via 
buffers/connector 352. Thus, a user who possesses several removable 
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memories 354 may replace a full removable memory 354 with an empty 
removable memory 354 to effectively expand the picture-taking capacity of 
camera 110. In the preferred embodiment of the present invention, removable 
memory 354 is typically implemented using a flash disk. 

Power supply 356 supplies operating power to the various components of 
camera 110. In the preferred embodiment, power supply 356 provides operating 
power to a main power bus 362 and also to a secondary power bus 364. The 
main power bus 362 provides power to imaging device 114, I/O 348, non-volatile 
memory 350 and removable memory 354. The secondary power bus 364 
provides power to power manager 342, CPU 344 and DRAM 346. 

Power supply 356 is connected to main batteries 358 and also to backup 
batteries 360. In the preferred embodiment, a camera 110 user may also 
connect power supply 356 to an external power source. During normal 
operation of power supply 356, the main batteries 358 provide operating power 
to power supply 356 which then provides the operating power to camera 1 1 0 via 
both main power bus 362 and secondary power bus 364. During a power failure 
mode in which the main batteries 358 have failed (when their output voltage has 
fallen below a minimum operational voltage level) the backup batteries 360 
provide operating power to power supply 356 which then provides the operating 
power only to the secondary power bus 364 of camera 110. 

FIGS. 4 and 5 are diagrams depicting the preferred hardware 
components of the camera's 110 user interface 408. FIG. 4 is back view of the 
camera 110 showing the LCD screen 402, a four-way navigation control button 
409, an overlay button 413, a menu button 414, and a set of programmable soft 
keys 416. FIG. 5 is a top view of the camera 110 showing a shutter button 418, 
and a mode dial 420. The camera may optionally include status LCD 406, 
status LCD scroll and select buttons 422 and 424, a sound record button 426, 
and zoom-in, zoom-out buttons 428a and 428b. 

The digital camera of the present invention is controlled by graphical- 
user-interface (GUI) based operating system (OS), which is in contrast to 
conventional digital cameras that are controlled by proprietary hardware 
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architectures. 

In the preferred embodiment of the present invention, the OS provides 
the digital camera with several different operating modes for supporting various 
camera functions. Although the digital camera may include several different 
operating modes, the modes relevant to this description are capture mode, and 
play mode. 

In capture mode, the camera 100 supports the actions of preparing to 
capture an image, and capturing an image through the use of either the LCD 
screen 402 or the status LCD 406. In play mode, the camera 110 supports the 
actions of displaying full-sized views of captured images, and play-backing 
various media types associated with the images, such as sound. The user may 
switch between the various modes, using the mode dial 420. When the camera 
is placed into a particular mode, that mode's default screen appears in the LCD 
screen 402 in which a set of mode-specific items, such as images, icons, and 
text, are displayed. 

The present invention provides a method and system for controlling user 
interaction in a digital imaging device using dynamic overlay bars. According to 
the present invention, the dynamic overlay bars are used to provide the user 
with both status information and interactive instructions. The interactive 
instructions are automatically updated in response to normal camera operations 
to guide the user through predefined operations of the camera, thus making the 
device extremely easy to use. In addition, the manner in which the dynamic 
overlay bars are displayed reduces viewing interference with the currently 
displayed object. 

Referring now to FIG. 6, a flow chart is shown illustrating the process of 
controlling user interaction in a digital imaging device using dynamic overlay 
bars in accordance with the present invention. The process begins by 
displaying an image on the LCD screen 402 along with at least one overlay bar 
that provides a dynamic prompt area in a way that minimizes viewing 
interference with the displayed image in step 450. 

In a preferred embodiment, viewing interference is minimized by 
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positioning the overlay bar along an edge of .the LCD screen 402 and by 
displaying the background of the bar translucently so that the user may see the 
image through the overlay bar. The overlay bar may also be displayed with a 
solid color background, but this is less desirable since the bar would overwrite 
that portion of the image. 

In response to the camera being placed into one of the operating modes, 
the overlay bar displays mode-specific information for the user in step 452. In a 
preferred embodiment, the mode-specific information displayed on the overlay 
bar includes a combination of static status information, dynamically updated soft 
key labels, and interactive instructions pertaining to the particular mode, as 
described further below. After the mode-specific information is displayed, the 
mode-specific information is then dynamically updated during the operation of 
the camera to guide the user through a mode-specific function in step 454. 

To more particularly describe the present invention, refer to FIGS. 7A and 
7B illustrating the use of dynamic overlay bars on the LCD screen 402 during 
two different operating modes of the digital camera 110. As shown, in a 
preferred embodiment of the present invention, two overlay bars 430 and 432 
are simultaneously displayed on the LCD screen 402, rather than one, to strike a 
balance between the amount of information provided to the user and the amount 
of screen area consumed by text and/or graphics. 

Overlay bar 430 may be used primarily to display status information and 
interactive instructions, while overlay bar 432 may be used primarily to display 
soft key labels 410 corresponding to soft keys 412. Both overlay bars 430 and 
432 may be turned-off in each of the camera operating modes by pressing the 
overlay "on/ofT button 413 so that users can have an unobstructed view of 
images if they so choose (off), or extra help in operating the camera (on). 

Referring to FIG. 7A, the display of the overlay bars 430 and 432 on the 
LCD screen 402 during capture mode is shown. In capture mode, the camera 
110 supports the actions of preparing to capture an image, and capturing an 
image through the use of either the LCD screen 402 alone or with the aid of an 
optional optical viewfinder (not shown). 
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Overlay bar 430 is updated with capture status information during capture 
mode, which may include a graphic memory gauge, and text indicating the state 
of the camera (Ready), for example. The memory gauge provides the user with 
a constant overview of camera memory usage in terms of disk space, and may 
also show working memory usage. In a preferred embodiment, the memory bar 
displays disk space usage as segments filling-up, and displays working memory 
usage as the bar below those segments, which is constantly updated to reflect 
current memory status. When the working memory buffers are empty, the 
bottom part of the bar would be clear. When there is the equivalent of storage 
for only a few pictures left, the storage gauge may flash and the overlay bar 430 
may be updated with a message, such as "Storage Almost Full", if a user tries 
to take a picture without adequate storage, then the overiay bar 430 may be 
updated to reflect this status by displaying the message "Inadequate Storage," 
along with an optional sound from the camera. 

The overlay bar 430 may also be updated to reflect other types of capture 
status information and may be expanded into additional lines if needed. The 
additional capture status information could include the following: 1) Low Battery 
Indication - when main batteries run low, a battery icon may replace the storage 
gauge and a overiay bar 430 may be updated to flash "Battery Low"; 2) Shake 
Warning Indication - when light level is too low for recommended hand held 
operation and user has disabled the strobe system "Shake Warning" may be 
displayed in the overiay bar 430; and 3) No Focus Indication - when the focus 
system cannot adequately focus the camera lens, a "No Focus" may be 
displayed in the overiay bar 430. 

Referring now to FIG. 7B, the display of the overiay bars 430 and 432 on 
the LCD screen 402 during play mode is shown. In a preferred embodiment, the 
play screen layout displays one full-sized image at a time and the user may 
chronologically scroll through the full-sized images in the LCD screen 402 using 
the left/right buttons on four-way navigation control button 409. Users can also 
play back various media types, such as time-lapse, bursts and slide show 
images according to either default or user defined play back rates. 
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In the play mode, overlay bar 430 displays status information relating to 
the current image being displayed, such as the image name/number, and the 
date and time of capture. The status information may also include graphical 
icons indicating what category of images the image belongs to and the image 
type. 

Referring to both FIGS. 7A and 7B, besides displaying status information, 
the second use of the dynamic overly bars of the present invention is to display 
soft key labels 410 for soft keys 412. As described in U.S. Patent Application 
No. 08/939,993 filed on September 26, 1997, entitled "A Method And System 
For Manipulating Images Stored In A Digital Imaging Device," assigned to the 
present assignee and hereby incorporated by reference, soft keys 412a, 412b, 
and 412c of the user interface 400 are programmable, i.e., they may be 
assigned predefined functions. The function currently assigned to a respective 
soft key 412 is indicated by the soft key labels 410a, 410b, and 410c displayed 
in overlay bar 432. After a soft key label 410 has been displayed, the user may 
then press the corresponding soft key 412 to have the function indicated by its 
label applied to the current image. 

Referring to FIG. 7B for example, the function assigned to the soft key 
412b in during play mode is a "Zoom" function, which allows a user to zoom in 
and out of a displayed image. When the user zooms-in on an image by pressing 
the soft key 412b, the "Zoom" soft key label 410b is changed to "Zoom-out". 
While an image is zoomed, the user may pan around the image using the four- 
way control button 406. 

The functions assigned to the soft keys 412, and thus the soft key labels 
410, are changed in response to several different factors. The soft keys 412 
may change automatically either in response to user actions, or based on 
predetermined conditions existing in the camera, such as the current operating 
mode, the image type, and so on. The soft keys 412 may also be changed 
manually by the user by pressing the menu button 415. Providing 
programmable soft keys 412 increases the number of functions that may be 
performed by the camera, while both minimizing the number of buttons required 
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on the user interface, and reducing the need to access hierarchical menus. 

As stated above, in addition to displaying status information and soft key 
labels, the dynamic overlay bars of the present invention may also be used to 
display interactive instructions to the user to guide user through camera 
functions. Basic types of camera functions include reviewing captured images, 
deleting images, annotating images with sound, and capturing groups of related 
images. With conventional cameras, the user would have to memorize 
complicated key sequences in order to perform these functions. 

The present invention, in contrast, uses the dynamic overlay bars to 
display interactive instructions that guide the user through operations such as 
adding sound to an image, deleting images and/or sound, and capturing groups 
of related images. 

As described in U.S. Patent Application No. 08/939,993, for example, after the 
user has captured an image and the image is displayed for review, the overlay 
bar 432 automatically reminds the user that he or she has the option to delete 
the image. That is, one of the soft key labels 410 is changed to "Delete" and the 
user may then delete image by pressing the corresponding "Delete" soft key 
412. 

Referring now to FIGS. 8A through 8C f diagrams illustrating how the 
overlay bars may be used to guide the user through a recording of a sound 
annotation are shown. The user may initiate the sound annotation function by 
pressing the record button 426 (see FIG. 5) while an image is displayed. In 
response, a record indication, such as a microphone icon, is automatically 
displayed in overlay bar 430 along with a display of the duration of the recording, 
as shown in FIG. 8A. After the sound annotation is recorded, the soft key labels 
410 may be updated to display three options "Play", "Delete", and "Save"; where 
"Play" plays back the recorded sound, "Delete" deletes the recorded sound, and 
"Save" saves the recorded sound. 

If the user is reviewing images in play or review modes, it is possible that 
the displayed image will have a sound annotation attached. Should the user 
presses the "Delete" soft key 412, it is unclear what operation the user wishes to 
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perform: delete the image, delete only the sound, or delete both. Indeed, an 
inexperienced user may not even consider all three of these possibilities before 
pressing the "Delete" button. Therefore, to guide the user through this 
operation, the dynamic overlay bars 430 and 432 are updated to prompt the 
user whether the image or the sound annotation is to be deleted, as shown in 
FIG. 8B. The user may then indicate which is to be deleted by pressing the 
corresponding soft key 412. 

While reviewing images, it is also possible that the user may press the 
record button 426. If the current image already includes a sound annotation, 
then it is unclear whether the user wishes to record a new sound annotation 
over the old one, or whether the user is unaware of the existing sound 
annotation. Therefore, to make sure the user doesn't inadvertently overwrite the 
existing sound, the overlay bar 430 is automatically updated to inform the user 
that sound will not be recorded until the user deletes the existing sound, as 
shown in FIG. 8C. In addition, if the user doesn't recall the contents of the 
previous sound annotation, the user may listen to it before deleting it by pressing 
"Play", or the user may cancel the record operation altogether by pressing "Exit". 
Thus, according to the present invention the user is enabled to perform complex 
tasks in the camera without fumbling through a set of hierarchical menus. 

Another use of displaying interactive instructions in the dynamic overlay 
bars 430 and 432 in accordance with the present invention is to direct the user 
through image capture sequences. The purpose of directed image capture 
sequences is to customize the camera's image capture process for a specific 
application. More specifically, a directed image capture is a camera feature that 
provides the user with interactive instructions and feedback during capture mode 
to guide the user through a series of task-oriented image captures. 

Upon initiation of a directed image capture sequence, interactive 
instructions are displayed the dynamic overlay bars 430and 432 that prompt the 
user to perform specific operations (capture image or capture sound), and for 
prompting the user to enter specific input (name and date). Customized directed 
image captures can be tailored to specific professions, such as insurance claims 
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adjusters and real estate agents, who would benefit from the use of a digital 
camera to capture groups of related pictures. 

Referring now to FIGS. 9A and 9B ? diagrams illustrating example directed 
image capture screens are shown. The example shown in FIG. 9A may pertain 
to an insurance-related directed image capture that prompts an insurance claims 
adjuster to take a series of pictures of a damaged vehicle, or it may pertain to a 
real estate application that guides a user through taking photos of a house for 
sale. 

In the insurance example, once the directed image capture has started, 
the user may be instructed to take various views of the damaged car. The user 
may also be shown the number of the current image in that sequence, and the 
total number of images to be captured. 

After the views of the car are taken, the directed image capture may then 
prompt the user to enter specific information, such as the name of the image, as 
shown in FIG. 9B. The user may then enter text by choosing letters using the 
four-way control button 409. For insurance purposes, the directed image 
capture may also request the user to input the owner's name, license plate 
number, claim number, and so on. The sequence of images and corresponding 
information may then be downloaded from the camera or to a host computer for 
automated database storage or web page generation. 

In one embodiment of the present invention, one or more directed image 
capture sequences may be provided in the camera as built-in functions, 
especially if the camera is tailored for specific industries. 

However, in a second aspect of the present invention, the camera is 
made more flexible by implementing the directed image capture sequences as a 
set of program instructions that are externally loaded into the camera. Once 
loaded in the camera 110, the instructions are then preferably executed by the 
GUI-based system software running on CPU 344. 

FIG. 10 is a block diagram illustrating the contents of ROM 350 where the 
software is stored, and DRAM 346 where the software is executed. - The 
software 600 may include a control application 602, a toolbox 604, drivers 612, a 
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kernel 614, and a startup/ configuration module 616. The control application 
602 is the main program that controls high-level functions of the digital camera 
and is responsible for interfacing with functions in the toolbox 604. 

Toolbox 604 comprises selected function modules that control how the 
digital camera captures and manipulates images. The modules may include 
image processors 606, a camera control shell 608, and a script interpreter 610. 
Image processors 606 are programs for enhancing (e.g., adjusting the contrast, 
sharpening, converting the image to gray-scale, etc.) the digital image received 
from imaging device 114. Camera control shell 608 receives and processes 
data structures for controlling camera functions. Script interpreter 610 translates 
and executes script statements, which are used to provide the directed image 
capture sequences and other camera 110 features, as explained below. 

Drivers 612 comprise program instructions for controlling various camera 
110 hardware components, such as motor 234 (FIG. 2) and a flash (not shown). 
Kernel 614 comprises program instructions providing basic underlying camera 
operating system services including synchronization routines, task creation, 
activation and deactivation routines, resource management routines, etc. 
Startup/configuration 616 comprises program instructions for providing initial 
camera 110 start-up routines such as the system boot routine and system 
diagnostics 

When the camera 110 is first turned on and booted up, the 
startup/configuration 616 module begins to execute and loads the drivers 612, 
the kernel 614, the control application 602, and system files containing 
configuration information into DRAM 346. Thereafter, operation of the camera is 
passed to the control application 602. In an alternative embodiment, the 
software 600 may executed out of ROM 350 in order to reduce the size of 
DRAM 346. 

The directed image capture sequence 618 may be loaded into the digital 
camera 110 from the removable memory 354 (FIG. 3), a host computer, or a 
network, and stored in DRAM 346 to run in place of the control application 602. 
in a preferred embodiment, the directed image capture sequence 618 is 
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implemented using a script, which is a program written with text-based 
commands that may be easily written by the user. As used herein, a script may 
be written in any intrepreted language, such as Basic and Lisp, for example. 

Once loaded into the camera, the script may be selected by the user from 
a menu where it is displayed for selection, and is thereafter executed by the 
control application 602 by passing the script to the script interpreter 610. The 
script interpreter 610 then translates and executes the script instructions 
comprising the directed image capture sequence 618 one-by-one. 

In an alternative embodiment, a directed image capture sequence 618 
may be implemented as a traditional application program, rather than a script. 
However, an application program is typically written by a software developer in a 
traditional computer language, such as C++, compiled, and stored in machine 
language, which is a more complicated process than adding new functions to 
the camera via a text-based interpreted script. 

FIG. 11 is a flow chart illustrating an exemplary process of installing and 
running a script-based directed image capture in a preferred embodiment of the 
present invention. The process begins by inserting the removable memory 354 
in step 700. When the removable memory 354 is installed, the removable 
memory 354 is mounted by the operating system 600 in step 702. Thereafter, 
the operating system searches for system files on the removable memory 354, 
which alert the digital camera 110 to the presence of an external program, in 
step 704. 

Any system files found on the removable memory 354 and corresponding 
directed image capture sequences 618 are then installed and made available to 
the user for selection via menu choices that appear on the LCD screen 402 in 
step 706. In a preferred embodiment, steps 704 and 706 are implemented as a 
hot-mount process when the removable memory 354 is inserted into the camera 

1 10, as described in U.S. patent application No. , entitled "Method 

And System For Dynamically Updating Software Functions In A Digital Capture 

Device (P1 49)," filed on , 1998, which is assigned to assignee of the 

present application and herein incorporated by reference. 
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Once the list of available directed image capture sequences 618 are 
displayed, the user selects one of the directed image capture sequences 618 to 
run in step 708. In a preferred embodiment, the list showing the available 
directed image capture sequences may be categorized in menus for easier 
selection. For example, assume a real estate agent has three different scripts 
for capturing images of different types of properties. The agent may name or 
create categories for the directed image capture sequences called "commercial", 
"industrial", and "residential", for instance. Selecting the residential category, for 
example, will cause a list of directed image captures to be displayed that are 
designed to capture pictures of different types of residential properties, such as 
one, two, and three bedroom homes. The user may then select a desired script 
depending on the particular house to be shot. 

In one preferred embodiment, the directed image capture selections 
displayed in the menus may be erased from the camera by rebooting the 
camera, or by removing the removable memory 354 from the camera 110. 

After the user selects one of the directed image capture sequences 618 
to run, the script interpreter 610 begins interpreting the directed image capture 
sequence 618 in step 710, and control is passed from the control application 
602 to the script. In step 712, the script interpreter 610 fetches the first 
command comprising the directed image capture sequence 618. 

It is then determined whether the fetched command is a script 
"WaitForShutter" command in step 714. This command causes control of the 
camera 110 to pass back to the control application 602 until the user presses 
the shutter button 418 to capture an image. The "WaitForShutter" command is 
preferably called with a quoted string parameter that is used in the dynamic 
overly bar 430 as the prompt to the user requesting an image capture (e.g., 
Take photo of kitchen"). 

If the command is a "WaitForShutter" command in step 714, then control 
is returned to the script after the user presses the shutter button 418 in step 716 
to capture an image. If the fetched command is not a "WaitForShutter" 
command in step 714, then the script interpreter 610 interprets and executes the 
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command in step 718. 

After the user presses the shutter button 418 or after a script command 
has been executed, it is determined rf the end of the script has been reached in 
step 720. If not, then the next command is fetched in step 712, and the process 
continues until the end of the script is reached, at which point control is returned 
to the control application 602 in step 722. 

Besides the "WaitForShutter" command, scripts may include two other 
categories of script commands. One category of commands pertain to camera 
settings, controls and other camera parameters specific to the subject and/or 
scene being captured, (ie: White Balance Modes, Exposure Modes, and Focus 
Modes). This category of commands enable users to input M Hints n optimizing the 
camera's photo systems for specific photographic conditions. 

The other category of commands may pertain to file system operations 
and image tagging functions specific to the way in which image data is stored in 
memory, (ie: Guided Capture, Prompted Text/Audio Annotation, and Automated 
Image Grouping/Cataloging/Indexing.) This category of commands is 
particularly useful when used in conjunction with desktop computer applications 
where the hosting application is coordinated to take advantage of the 
Preformatted media organization and tag information. For example, while a 
directed image capture sequence guides the user though a series of steps to 
create an image grouping, the script commands comprising the sequence 
generate appropriate tags and data structures to group the images and text 
captured during the sequence. 

No matter whether the dynamic overlay bars of the present invention are 
used to display status information, soft key labels, or interactive instructions, as 
described herein, one important component affecting the user's experience is 
the method used to display the overlay bars on the image. 

One approach would be the follow prior art techniques for displaying text 
(e.g. image name) over an image. This approach typically includes the following 
steps: 1) fetching the image to be displayed, which is typically stored in JPEG 
format, 2) decompressing and resizing the image, 3) displaying the 
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decompressed image block-by-block, and then after the image is fully displayed, 
4) writing the text on top of the image. 

The problem with this method is that is visually unappealing to the user, 
and it reduces the performance of camera when the user turns-off the text 
display while viewing the image. The reason the method reduces camera 
performance is the following. When text or graphics are displayed over the 
image, they obscure a portion of the image. And when the text is turned-off, the 
obscured portions of the image must be displayed so that the original image is 
seen without the text. In order to do this, however, the entire JPEG image must 
be fetched and decompressed again so that the obscured portions of the image 
can be displayed on the LCD, which can be a time consuming operation. 

A third aspect of the present invention overcomes these disadvantages 
by providing an improved method and system for displaying the overlay bars that 
not only enhances the visual effect associated with the overlay bars, but also 
eliminates the need to re-decompress the JPEG image data when the user 
turns-off the overlay bars, thereby increasing performance of the camera. 

According to this aspect of the present invention, the overlay bars are 
displayed first, followed by the image, wherein the image is made to appear as 
though it is sliding underneath the overlay bars as it is being displayed. The 
image appears as though is it is sliding underneath the overlay bars because the 
image is displayed on the LCD screen 402 line-by-line or block-by-block (as 
used herein, a block may include anywhere from one line to sixteen lines of 
image data). As the display of the image progresses from the top of the screen 
402, the image therefore appears to be displayed behind the overlay bars 430 
and 432 which are already present on the LCD screen 402. 

The overlay bars 430 and 432 are also provided with a translucent 
background so that so that the overlay bars 430 and 432 themselves do not 
obscure the image, but the text is easily distinguishable from the colors of the 
displayed image. The result is that after the image has been displayed, the 
overlay bars appear as a separate layer over the image. Further, the portions of 
the original image that intersect with the overlay bars 430 and 432 are saved, so 
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that when the user tums-off the overlay bars 430 and 432, only these portions of 
the image are redisplayed to restore the image. Thus this aspect of the present 
invention eliminates the need to re-decompress and display the entire image 
again, thereby increasing system performance. 

Where typically, specialized hardware would be required to achieve the 
above-described effects, the present invention accomplishes the task through 
software and the manipulation of several memory buffers, as shown in FIG. 12A. 

FIG. 12A is a diagram illustrating a buffer organization for displaying 
overlay bars, which in a preferred embodiment, resides in DRAM 346, The 
buffer organization includes an overlay bar buffer 540, a backstore buffer 542, 
and a display buffer 544. According to the present invention, the overlay bar 
buffer 540 is used to store the graphics data (graphics and text) that will be 
displayed in the overlay bars 430 and 432. In a preferred embodiment the 
overlay bar buffer 540 is divided into a top and bottom portion, which store 
twenty lines of data each that correspond to the top and bottom overlay bar 430 
and 432, respectively. 

The backstore buffer 542 is used to store original image data 
corresponding to the area of the LCD screen 402 where the overlay bars 430 
and 432 will be displayed. The backstore buffer 542 is also divided into a top 
and bottom portion that are the same size as the top and bottom portions of the 
overlay bar buffer 540. 

As is typical in most rendering systems, the display buffer 544 is used to 
store the actual data that is to be displayed on the LCD. The data in the display 
buffer is accessed by LCD controller 390 (FIG. 3) and displayed on the LCD. 

FIG. 12B is a flow chart illustrating the process of displaying overlay bars 
on the LCD in accordance with the present invention. The first step in the 
process is to preferably receive an input line of decompressed image data from 
an image processing system in step 800. The process may also be modified to 
receive an input block of decompressed image data. In a preferred 
embodiment, the image processing system for providing the input data may 
include an image decompressor for decompressing the image data, and a 
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resizer for resizing the lines of image data to fit the size of LCD screen 402. 

Next, it is determined whether the overlay bars 430 and 432 are tumed- 
on or off in step 802. If the overlay bars are turned-off, then the line of image 
data is copied directly to the display buffer 544 in step 804 for display on the 
LCD screen 402 and the process continues. If the overlay bars remain off for 
the duration of the time it takes to display the image line-by-line or block-by- 
block, then the entire image is displayed on the LCD screen 402 using only the 
display buffer 544. 

If the overlay bars are turned-on in step 802, then it is determined 
whether the line of data will be displayed within the area of the LCD screen 402 
that is occupied by an overlay bar in step 806. If the line is within an overlay 
bar, the line is copied into the backstore buffer 542 in step 808. The purpose of 
copying the line to the backstore buffer 542 is to save the portion of the image 
that will be displayed underneath the overlay bars 430 and 432. 

After the current line of image data is copied into the backstore buffer 
542, the corresponding line stored in the overlay bar buffer 540 is merged with 
the current line in the backstore buffer 542 in step 810. The purpose of merging 
the two lines is to display the background of the overlay bars 430 and 432 
translucently over the image on the LCD screen 402. This is done by halving 
the luminance value of each pixel of the image data from the backstore buffer 
542 that falls within the bounds of an overlay bar 430 or 432, and overwriting 
each pixel in the line of image data that falls under a pixel of text or graphic data 
from the overlay bar buffer 540. Halving the luminance value of the image data 
causes the colors of the image that overlap an overlay bar 430 or 432 to be half 
as bright, thus giving the overlay bar 430 or 432 a translucent appearance and 
allowing the user to see the image through the overlay bar 430 or 432, as shown 
in FIG. 7B. In an alternative embodiment, the translucency of the overlay bars 
430 and 432 is provided by increasing, rather than decreasing, the luminance 
value of each image pixel failing within the area of an overlay bar. In this case, 
the text displayed in the overlay bars 430 and 432 is displayed using a dark 
color. 
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As the line from the overlay bar buffer 540 is merged with the line from 
the backstore buffer 542, the resulting merged line is written into the display 
buffer 544 for display in step 812. If the current line is the last line of image data 
in step 814, then the process ends. Otherwise the next line of image data is 
received in step 800 and the process continues. In an alternate embodiment of 
the present invention, this determination of whether the overlay bars 802 are 
on/off in step 802 may be performed after copying the input line to the backstore 
buffer 542 in step 8. In this embodiment, the input line is copied into the 
backstore buffer 542 even when the overlay bars 430 and 432 are off. 

In a preferred embodiment of present invention, the software 600 
controlling the digital camera 110 is implemented as event driven software, 
which responds to input from the user (select menu, press button, etc.) or other 
applications at unregulated times. When, for example, the user first switches to 
play mode and/or selects a new image to display, the first steps that are 
performed in the process are to blank the LCD screen 402, fill the overlay bar 
buffer 540 with relevant mode-specific information, and then contents of the 
overlay bar buffer 540 and the backstore buffer 542 are merged and written to 
the display buffer 544. In this case, the backstore buffer 542 may contain black 
or white pixel values to provide the blank screen. Thereafter, the process 
proceed as described in FIG. 13. 

If the user turns-off the overlay bars 430 and 432 while an image is 
displayed, then the process is interrupted and software 600 copies the entire 
contents of the backstore buffer 542, which contains the original image data, to 
the display buffer 544 for display. This causes the overlay bars to disappear 
from the LCD screen 402 and restores the original image without having to re- 
decompress and display the entire image over again. 

If the user then turns-on the overlay bars 430 and 432, the software 600 
merges the contents of the overlay bar buffer 540 and the backstore buffer 542 
to provide the translucent bars and text over the image, and then copies the 
result to the display buffer 544 for display. This may be done by executing step 
812 and 814 for each line of the data in the buffers 540 and 542. 
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Also, when the overlay bars 430 and 432 are on, if the overlay bars 430 
and 432 are updated by the control application 602 due to a change in status or 
instructions, the contents of the overlay bar buffer 540 and the backstore buffer 
542 are remerged and written into the display buffer 544 for display. 

A method and system for controlling user interaction in a digital imaging 
device using dynamic overlay bars has been disclosed, which enables a user to 
apply camera functions and features to images with minimum effort, while 
allowing for easy viewing of the image. In addition, the present invention 
displays dynamic interactive instructions to the user in the form of directed 
image capture to guide the user through complex task, without the need for help 
screens or for the user to remember complicated key sequences. Finally, the 
present invention provides a method for displaying overlay bars that eliminates 
the need to re-decompress and display the image when the user turns-off the 
overlay bars, which increases the responsiveness of the camera. 

Although the present invention has been described in accordance with 
the embodiments shown, one of ordinary skill in the art will readily recognize that 
there could be variations to the embodiments and those variations would be 
within the spirit and scope of the present invention. For example, the functions 
assigned to the soft keys, the number of soft keys, and the placement of the soft 
keys and labels in and around the display may vary. The method and system may 
also be implemented in digital imaging devices having only two modes, but that 
have multiple navigation screens within the "play mode" Accordingly, many 
modifications may be made by one of ordinary skill in the art without departing 
from the spirit and scope of the appended claims. 
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CLAIMS 

What is claimed is: 

1 A method for controlling user interaction in a digital imaging device, 
the digital imaging device having a display the method comprising the steps of: 

a) providing the digital imaging device with a directed image capture 
sequence comprising a set of program instructions; 

b) executing the directed image capture sequence to display 
interactive instructions on the display that prompt the user to 
perform specific operations; and 

c) in response to the user performing the specific operations, 
automatically updating the interactive instructions, thereby guiding 
the user through a series of related image captures. 

2 The method of claim 1 wherein step a) further includes the step of 
providing the directed image capture sequence by externally loading the 
program instructions into the digital imaging device. 

3 The method of claim 2 wherein step a) further includes the step of 
providing the program instructions as a text-based script 

4 The method of claim 3 wherein step b) further includes the step of 
executing the directed image capture sequence by interpreting the text-based 
script. 

5 The method of claim 4 wherein step c) further includes the step of 
prompting the user for specific information, and entering the specific information 
on a text entry screen. 

6 The method of claim 5 further includes the step of displaying the 
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interactive instructions on a translucent overlay bar. 

7 A method for directing image capture sequences in a digital 
imaging device having a display, the method comprising the steps of: 

a) externally loading a script comprising program commands into the 
digital imaging device; 

b) displaying the script as a menu item for selection by a user; 

c) in response to the user selecting the script menu item from the 
menu, passing operational control from the digital imaging device 
to a script interpreter; 

d) interpreting and executing each of the script commands, wherein a 
first plurality of the script commands are for displaying interactive 
instructions on the display requesting the user to perform specific 
camera operations, thereby guiding the user through a series of 
image captures; 

e) passing operational control from the script interpreter to the digital 
imaging device after the script has requested the user to capture 
an image; and 

f) repeating steps d-f until the script completes execution. 

8 The method of claim 7 wherein step a) further includes the step of loading 
the into the digital imaging device from a removable memory. 

9 The method of claim 8 wherein step e) further includes the step of 
passing operational control from the digital imaging device to the script 
after the user has captured the image. 

10 The method of claim 9 wherein step f) further includes the step of passing 
operational control back to the digital imaging device after the script 
completes execution. 
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11 An digital imaging device comprising: 
an imaging device for capturing image data; 

a memory coupled to the imaging device for storing the image data as captured 
images; 

a display for displaying a captured image; 

means for externally loading a script comprising program commands into the 
memory; 

a processor coupled to the imaging device and to the memory for controlling 
operation "of the digital imaging device, the processor including means for 
interpreting and executing each of the script commands, wherein when the script 
commands are executed, interactive instructions are displayed on the display 
requesting the user to perform specific camera operations, thereby guiding the 
user through a series of related image captures. 

12 A digital imaging device as in claim 1 1 wherein the means for 
interpreting and executing each of the script commands comprises a script 
interpreter. 

13 A digital imaging device as in claim 12 wherein operation control of 
the digital imaging device is controlled by a control program stored in memory 
and executed by the processor, the control program comprising, 

means for displaying the script as a menu item on the display for selection by a 
user; and 

means for passing operational control to the script interpreter in response to the 
user selecting the script menu item from the menu. 
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